Use `GaussianRandomFields` to calculate optimal proposal covariance #216

matt-graham · 2022-01-27T10:00:15Z

Fixes #213 and fixes #214.

Updates the covariance method in src/OptimalFilter.jl to use the apply method in the GaussianRandomFields package rather than hardcoding a particular covariance function. The expected interface of a model is also slightly updated to have get_model_noise_params return a structure defining the parameters of the covariance function. These changes both ensure the covariance function definition used in sampling the state noise for the model and assumed in the optimal proposal distribution match (which may not currently be the case as described in #213) and means the optimal proposal distribution directly generalises to any isotropic covariance function defined in the GaussianRandomFields package.

I also updated some of the comments in src/OptimalFilter.jl to clarify the match to the equations / notation in Dietrich and Newsam (1996), including specifically resolving the TODO described in #214.

src/OptimalFilter.jl

matt-graham · 2022-01-27T11:22:41Z

Turns out I had not being using my local development version of ParticleDA.jl when running the test locally 🤦 which unsurprisingly meant all the tests were passing. The failing unit tests I think just need me to update these lines

ParticleDA.jl/test/runtests.jl

Line 296 in c278bbc

    
           noise_params = (sigma = model_params.sigma[1], lambda = model_params.lambda[1], nu = model_params.nu[1])

ParticleDA.jl/test/runtests.jl

Line 366 in c278bbc

    
           noise_params = (sigma = model_params.sigma[1], lambda = model_params.lambda[1], nu = model_params.nu[1])

to reflect that noise_params is now expected to be a GaussianRandomFields covariance structure.

The optimal filter integration test also fails due to mismatch from the reference values. If there was a mismatch between the covariance being assumed in the state noise generation and optimal proposal distribution implementation that would possibly explain this. I'll check whether the values we are getting from the covariance method have changed, and if so regenerate the reference data.

Accounts for fix to covariance function definition in optimal proposal

matt-graham · 2022-01-27T14:56:29Z

I've updated the tests to fix the problems described above, plus adjusting some manual covariance function definitions in optimal_filter_validation.jl and runtests.jl to be consistent with the Matern covariance function being used (specifically using the simplified form of the Matern covariance for ν=0.5 as the test case).

I also noticed what I think is a bug in GaussianRandomFields which means the σ parameter is being ignored when evaluating covariance matrices using apply. I have raised an issue about this: PieterjanRobbe/GaussianRandomFields.jl/issues/45. For now I have manually multiplied the output of apply by σ^2 to give the correctly scaled covariances. If this is confirmed as a bug and fixed upstream in GaussianRandomFields we will need to remove this manual scaling.

test/optimal_filter_validation.jl

giordano · 2022-01-27T15:51:18Z

test/optimal_filter_validation.yaml

@@ -11,4 +11,4 @@ model:
    obs_noise_std: 0.01
    sigma: 1.0
    lambda: 1.0
-    nu: 2.5
+    nu: 0.5


Any particular reason for changing this?

This is what I was referring to with

to be consistent with the Matern covariance function being used (specifically using the simplified form of the Matern covariance for ν=0.5 as the test case)

As the Matern covariance function is particularly simple for nu=0.5 I changed to using this value so that in the optimal_filter_validation.jl definition of R

ParticleDA.jl/test/optimal_filter_validation.jl

Lines 99 to 105 in 1f3e8e3

function R(x::Float64, y::Float64, th)

Cov = (th.s^2)*exp(-norm([x, y])/th.l)

return(Cov)

end

we can just use a simple exponential expression rather than having to use SpecialFunctions to load the gamma and besselk functions. It might still be better to do this though I guess from a robustness standpoint so that the test is valid for any value of nu. Or at least I should perhaps put a comment in the YAML file to make clear this value shouldn't be changed?

we can just use a simple exponential expression rather than having to use SpecialFunctions to load the gamma and besselk functions.

If the worry is to load another package during the tests, SpecialFunctions is already pulled in by GaussianRandomFields, so it isn't much more work.

It might still be better to do this though I guess from a robustness standpoint so that the test is valid for any value of nu.

I think that'd be better, yes.

Or at least I should perhaps put a comment in the YAML file to make clear this value shouldn't be changed?

At very least this, yes, please.

src/ParticleDA.jl

Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>

giordano

Looks great to me, after deciding what value of nu to use in the tests

tkoskela · 2022-01-28T16:32:21Z

test/optimal_filter_validation.jl

-  Cov = (th.s^2)*exp(-(abs(x)+abs(y))/(2.0*th.l))
+  Cov = (th.s^2)*exp(-hypot(x, y) / th.l)


These two expressions don't look equal, was this an intentional correction?

See discussion above #216 (comment) 😉

Ah! I didn't notice the resolved discussion

tkoskela · 2022-01-28T16:34:19Z

src/OptimalFilter.jl

-# Covariance between observations R_22, from equationd 3-4 in in Dietrich & Newsam 96
-# TODO: Ask Alex why we add sigma^2 on the diagonal
-function covariance_stations!(cov::AbstractMatrix{T}, grid::Grid, stations::NamedTuple, noise_params::NamedTuple, std::T) where T
+# Covariance between observations R̅₂₂ = R₂₂ + Σ, where R₂₂ is as defined in equations 3 and 4 in
+# Dietrich & Newsam 96 and Σ is the observation noise covariance (see note at end of
+# Section 2 in Dietrich & Newsam 1996)
+function covariance_stations!(cov::AbstractMatrix{T}, grid::Grid, stations::NamedTuple, covariance_structure::IsotropicCovarianceStructure{T}, std::T) where T


Thanks for catching and fixing the TODO 😄

Project.toml

matt-graham · 2022-01-31T13:55:00Z

I've now put the full definition of the Matern covariance function in optimal_filter_validation.jl so that the corresponding test should work for any nu value used in the YAML file and changed back the nu value used in the YAML file to the previous value of 2.5.

giordano · 2022-01-31T14:11:27Z

Looks perfect! Thanks a lot again!

Use GaussianRandomFields to calculate optimal proposal covariance

0455f6a

giordano reviewed Jan 27, 2022

View reviewed changes

src/OptimalFilter.jl Outdated Show resolved Hide resolved

matt-graham added 5 commits January 27, 2022 21:12

Updating reference data for integration tests

21565b3

Accounts for fix to covariance function definition in optimal proposal

Fixing noise_params type in optimal filter unit test

de72fd3

Adjusting covariance function parameters and definition in tests

d001345

Manually scale covariance function by sigma squared

d9bbaca

Merge branch 'master' into mmg/optimal-proposal-grf-covariance

1f3e8e3

giordano reviewed Jan 27, 2022

View reviewed changes

matt-graham and others added 2 commits January 28, 2022 09:25

Use hypot instead of norm

37e68f8

Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>

Update type signature in docstring of get_model_noise

7538aef

Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>

giordano requested a review from tkoskela January 28, 2022 12:10

giordano approved these changes Jan 28, 2022

View reviewed changes

tkoskela reviewed Jan 28, 2022

View reviewed changes

tkoskela approved these changes Jan 28, 2022

View reviewed changes

Generalizing optimal filter validation covariance function

7621968

giordano reviewed Jan 31, 2022

View reviewed changes

Project.toml Outdated Show resolved Hide resolved

Remove incorrectly added StableRNGs dependency

ffc4182

matt-graham merged commit fbb3628 into Team-RADDISH:master Jan 31, 2022

matt-graham deleted the mmg/optimal-proposal-grf-covariance branch January 31, 2022 14:49

tkoskela mentioned this pull request Apr 25, 2022

Generalising the optimal proposal implementation #218

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `GaussianRandomFields` to calculate optimal proposal covariance #216

Use `GaussianRandomFields` to calculate optimal proposal covariance #216

matt-graham commented Jan 27, 2022

matt-graham commented Jan 27, 2022

matt-graham commented Jan 27, 2022

giordano Jan 27, 2022

matt-graham Jan 28, 2022

giordano Jan 28, 2022

giordano left a comment

tkoskela Jan 28, 2022

giordano Jan 28, 2022

tkoskela Jan 28, 2022

tkoskela Jan 28, 2022

matt-graham commented Jan 31, 2022

giordano commented Jan 31, 2022

	function R(x::Float64, y::Float64, th)

	Cov = (th.s^2)*exp(-norm([x, y])/th.l)

	return(Cov)

	end

		Cov = (th.s^2)exp(-(abs(x)+abs(y))/(2.0th.l))
		Cov = (th.s^2)*exp(-hypot(x, y) / th.l)

Use GaussianRandomFields to calculate optimal proposal covariance #216

Use GaussianRandomFields to calculate optimal proposal covariance #216

Conversation

matt-graham commented Jan 27, 2022

matt-graham commented Jan 27, 2022

matt-graham commented Jan 27, 2022

giordano Jan 27, 2022

Choose a reason for hiding this comment

matt-graham Jan 28, 2022

Choose a reason for hiding this comment

giordano Jan 28, 2022

Choose a reason for hiding this comment

giordano left a comment

Choose a reason for hiding this comment

tkoskela Jan 28, 2022

Choose a reason for hiding this comment

giordano Jan 28, 2022

Choose a reason for hiding this comment

tkoskela Jan 28, 2022

Choose a reason for hiding this comment

tkoskela Jan 28, 2022

Choose a reason for hiding this comment

matt-graham commented Jan 31, 2022

giordano commented Jan 31, 2022

Use `GaussianRandomFields` to calculate optimal proposal covariance #216

Use `GaussianRandomFields` to calculate optimal proposal covariance #216